In one conventional method, an image is scaled, and the generated scaled image undergoes a known discrete cosine transform (DCT) process and quantization process. Some DCT coefficients are extracted from the low-frequency component side of DCT coefficients obtained by the quantization result, and are set as an image feature amount of an original image, which are used in image search (ISO/IEC JTC1/SC29/WG11/W3522 “MPEG-7 Visual Working Draft 4.0” (to be referred to as [VWD4.0] hereinafter) or ISO/IEC JTC1/SC29/WG11/N3521 “MPEG-7 Visual part of eXperimentation Model Version 7.0” (to be referred to as [VXM7.0] hereinafter)).
An image feature amount extraction process in [Color Layout Descriptor] described in [VWD4.0] or [VXM7.0] will be described below using FIGS. 11 to 15.
FIG. 11 is a view for explaining an image feature amount extraction process in [Color Layout Descriptor] described in [VWD4.0] or [VXM7.0]. FIG. 12 is a flow chart showing that image feature amount extraction process.
An image scaling process for scaling an original image 10001 to 8×8 pixels for respective R, G, and B color components is executed (step S10401). A color conversion process for converting pixels of generated 8×8 pixel images 10011, 10012, and 10013 for respective color components into color component data 10021, 10022, and 10023 on a YCbCr color space is executed (step S10402).
The color component data 10021, 10022, and 10023 on the YCbCr color space undergo a DCT process to acquire DCT coefficients 10031, 10032, and 10033 for respective color components (step S10403).
Note that the image scaling process, color conversion process, and DCT process can be known ones.
The DCT coefficients 10031, 10032, and 10033 undergo a quantization process (step S10404).
According to, e.g., [VWD4.0], the quantization process can be implemented by processes described by program codes 10000 to 10003 below. The program codes 10000 to 10003 are described using known C language.
According to [VWD4.0], since different quantization processes are executed for DC and AC components for each of Y and Cb/Cr components, the program codes 10000 to 10003 that implement four different processes are prepared.
Program code 10000: Quantization process for DC component of Y coefficient
Int quant_Y_DC(int i){ Int j; i=i/8; if(i>192) j=112+(i−192)/4; else if(i>160) j=96+(i−160)/2; else if(i>96) j=32+i−96; else if(i>64) j=16+(i−64)/2; else j=i/4; return j>>1;}
Program code 10001: Quantization process for DC component of Cb/Cr coefficient
Int quant_CbCr_DC(int i){ int j; i=i/8; if(i>191) j=63; else if(i>160) j=56+(i−160)/4; else if(i>144) j=48+(i−144)/2; else if(i>112) j=16+i−112; else if(i>96) j=8+(i−96)/2; else if(i>64) j=(i−64)/4; else j=i/0; return j;}
Program code 10002: Quantization process for AC component of Y coefficient
Int quant_Y_AC(int i){ int j; i=i/2; if(i>255) i=255; if(i<−256) i=−256; if(abs(i)>127) j=64+abs(i)/4; else if(abs(i)>63) j=32+abs(i)/2; else j=abs(i); j=(i<0)?−j:j; return(int)trunc(((double)j+128.0)/8.0+0.5);}
Program code 10003: Quantization process for AC component of Cb/Cr coefficient
Int quant_CbCr_AC(int i){ int j; if(i>255) i=255; if(i<−256) i=−256; if(abs(i)>127) j=64+abs(i)/4; else if(abs(i)>63) j=32+abs(i)/2; else j=abs(i); j=(i<0)?−j:j; return(int)trunc(((double) j+128.0)/8.0+0.5);}
As a result of the quantization processes described by the program codes 10000 and 10001, the DCT coefficient of the Y or Cb/Cr component that has undergone quantization (to be referred to as quantized DCT coefficients hereinafter) assumes a value ranging from 0 to 63. On the other hand, as a result of the quantization processes described by the program codes 10002 and 10003, the quantized DCT coefficient of the Y or Cb/Cr component assumes a value ranging from 0 to 31. Hence, the DC component of the Y or Cb/Cr component can be expressed by 6 bits (unsigned), and the AC component can be expressed by 5 bits (unsigned).
A coefficient select process for selecting some coefficients from the low-frequency component side of quantized DCT coefficients 10041, 10042, and 10043 obtained as a result of the quantization processes is executed (step S10405).
In case of FIG. 11, six quantized DCT coefficients of the Y component, and three each quantized DCT coefficients of the Cb/Cr components are selected. In practice, the coefficient select process is implemented by linearly rearranging quantized DCT coefficients, which are arranged in an 8×8 two-dimensional matrix by a zigzag scan process shown in FIG. 13, and selecting some coefficients from the head of the rearranged coefficients. Reference numeral 10101 in FIG. 13 denotes a chart indicating the zigzag scan, and a numerical value from 1 to 64 described in each of 8×8 blocks indicates the order of position of that quantized DCT coefficient from the head after the coefficients are linearly rearranged.
Some quantized DCT coefficients are acquired from the low-frequency component side of the obtained DCT quantized coefficients. According to [VWD4.0], the number of quantized DCT coefficients to be acquired is one of 1, 3, 6, 10, 15, 21, 28, and 64. The numbers of quantized DCT coefficients of the Cb and Cr components are equal to each other, but the numbers of quantized DCT coefficients of the Y and Cb/Cr components can be independently set.
As a default, six quantized DCT coefficients of the Y component, and three each quantized DCT coefficients of the Cb/Cr components are selected. In FIG. 11, default coefficient selection is made, i.e., six quantized DCT coefficients (10051) of the Y component, and three each quantized DCT coefficients (10052, 10053) of the Cb/Cr components are selected.
The selected quantized DCT coefficients 10051, 10052, and 10053 are defined as an image feature amount, i.e., Color Layout descriptor of the original image 10001.
According to [VWD4.0], this Color Layout descriptor is stored in a binary structure, as shown in FIG. 14 or 15. In a structure 10201 in FIG. 14 or 10301 in FIG. 15, each block expresses one bit. In the structure 10201 or 10301, a flag and coefficients are partitioned in units of fields for the sake of convenience. However, in practice, these flag and coefficients are continuously stored in the order indicated by the broken arrows in FIG. 14 or 15.
FIG. 14 shows the binary structure when Color Layout descriptor is a default, i.e., when six quantized DCT coefficients of the Y component, and three each quantized DCT coefficients of the Cb/Cr components are selected. In this case, “0” is stored in an expansion flag at the head of the structure. Following the flag, six quantized DCT coefficients of the Y component, three quantized DCT coefficients of the Cb component, and three quantized DCT coefficients of the Cr component are stored in turn. Since the DC component of a quantized DCT coefficient can be expressed by 6 bits (unsigned) and the AC component can be expressed by 5 bits (unsigned), each DC component is stored in a 6-bit field, and each AC component is stored in a 5-bit field.
On the other hand, FIG. 15 shows the binary structure when Color Layout descriptor is not a default. In this case, “1” is stored in an expansion flag at the head of the structure. Two 3-bit fields follow the flag. These 3-bit fields are coefficient designation fields used to indicate the numbers of quantized DCT coefficients of the Y and Cb/Cr components. For example, the following table shows correspondence between bit patterns indicating such numbers and the numbers of quantized DCT coefficients.
Number of quantized DCTBit patterncoefficients0001001301060111010015101211102811164
Furthermore, after these two 3-bit coefficient designation fields, quantized DCT coefficients are stored in the order of Y, Cb, and Cr components. The numbers of coefficients of Y, Cb, and Cr components are designated by the coefficient designation fields. In the structure 10301 in FIG. 15, for example, six quantized DCT coefficients of the Y component and six each quantized DCT coefficients of the Cb/Cr component are stored.
According to [VXM7.0], a similarity between images based on the quantized DCT coefficients selected as the image feature amount is calculated by a similarity calculation formula below. For example, similarity D between two Color Layout descriptors CLD1 (YCoeff, CbCoeff, CrCoeff) and CLD2 (YCoeff′, CbCoeff′, CrCoeff′) is computed by:
                    D        =                ⁢                                                            ∑                                  i                  =                  0                                                                      Max                    ⁢                                          {                      NumberOfYCoeff                      }                                                        -                  1                                            ⁢                                                                    λ                    Yi                                    ⁡                                      (                                                                  YCoeff                        ⁡                                                  [                          i                          ]                                                                    -                                                                        Ycoeff                          ′                                                ⁡                                                  [                          i                          ]                                                                                      )                                                  2                                              +                                                ⁢                                                            ∑                                  i                  =                  0                                                                      Max                    ⁢                                          {                      NumberOfCCoeff                      }                                                        -                  1                                            ⁢                                                                    λ                    Cbi                                    ⁡                                      (                                                                  CbCoeff                        ⁡                                                  [                          i                          ]                                                                    -                                                                        Cbcoeff                          ′                                                ⁡                                                  [                          i                          ]                                                                                      )                                                  2                                              +                                                ⁢                                            ∑                              i                =                0                                                              Max                  ⁢                                      {                    NumberOfCCoeff                    }                                                  -                1                                      ⁢                                                            λ                  Cri                                ⁡                                  (                                                            CrCoeff                      ⁡                                              [                        i                        ]                                                              -                                                                  Crcoeff                        ′                                            ⁡                                              [                        i                        ]                                                                              )                                            2                                          where λ is a weighting value for each quantized DCT coefficient, and [VXM7.0] shows the following weighting value table. Note that the weighting value in a blank of the table is 1. Also, weighting values for those which are not shown in this table (the order is 7 or later) are 1.
Order of coefficients123456Y222111Cb211Cr422
When two Color Layout descriptors have different numbers of quantized DCT coefficients, the similarity calculation formula is applied in correspondence with fewer quantized DCT coefficients, or deficient quantized DCT coefficients are complemented with values “16”, and the similarity calculation formula is applied in correspondence with more quantized DCT coefficients.
However, the image similarity calculation formula according to [VMX7.0] alone cannot identify images which have undergone one or both of:                image rotation        mirror conversion of image.        
A person can easily recognize that a given image and another image are similar to each other although they are reversed in the horizontal direction, or are similar to each other if one of them is rotated 90°. However, the similarity calculation formula of Color Layout descriptors described in [VXM7.0] cannot make such recognition.